Persistent L1 Writer

Introduction

The persistent L1 writer is the component that provides a stateful interface to simplify the coordination of the multiple L1Writerinstances.

Use case diagram

1. The user (wallet service) calls embedData with arbitrary data to embed (e.g. an L2-transaction) and the service replies with a uuid.
2. The user can use the uuid to check the status of the transaction:
   • PENDING: the transaction has not been processed. It is probably waiting for other transactions to be batched and compressed.
   • MEMPOOL: the transaction has been sent to the node. There is a high probability that the transaction is eventually included in a block, however, it may not be the case!
   • CONFIRMED: the transaction has been confirmed on the blockchain.

WARNING Be aware that a reorganization may delete your transaction.

Implementation

    embedData(data: [Byte]!, network: Network!): ID

    status(uuid: ID!): Status

    type StatusResult {
      txHash: ID, // Optional
      status: EmbedStatus
    }

    enum EmbedStatus {
      PENDING,
      MEMPOOL,
      CONFIRMED,
    }

The implementation requires a database to store the sent transaction, and their status.

1. The User (wallet service) calls embedData(data, network)
2. The Writer generates a uuid and stores the data, the uuid, the network, and the status=PENDING in the database.
3. The User receives the uuid that should store in a persistent storage.
4. The User (wallet service) calls status(uuid).
5. The Writer returns the status from the database of the transaction.

In the background, the l1_writer will do the following in a loop (notice, that these tasks can be done in parallel because they are mutually exclusive tasks):

• Get all PENDING transactions:
  • Call prepare_transaction for each transaction (or in the future, batch them together and call PreparedTransaction with the compressed batch).
  • Add the PreparedTransaction to the database row of the pending transaction.
  • Call query_or_write_transaction with the PreparedTransaction:
    • Success (Sent): update the status to MEMPOOL and insert the txHash
      • Status: Mempool and Confirmed should never happen because we status=PENDING is attached to embed tasks that are new or that have failed InvalidPreparedTransaction (see next bullet point).
    • Fail (InvalidPreparedTransaction): remove the PreparedTransaction from the database. Wait for the next loop to try again.
• Get all MEMPOOL transactions (these are transactions that have been Sent):
  • Check the status calling query_or_write_transaction:
    • If the status is InvalidPreparedTransaction, change the status to PENDING, and set the preparedTransaction column to null.
    • If the status is Sent/Mempool, do nothing. The transaction in the mempool will eventually be mined and Confirmed.
    • If the status is Confirmed, update the status to CONFIRMED.
• CONFIRMED transactions will be confirmed forever. If there is a reorganization on the chain and our transaction is removed, it is the responsibility of the client to call embed again.
• (optional) You may (or not) want to eventually delete CONFIRMED transactions.

Optional features

• The service can run in parallel with multiple replicas on k8s. This would require careful treatment of the database rows when processing:

        BEGIN;

        SELECT uuid, prepared_transaction
          FROM table_xxx
          WHERE status = PENDING
          LIMIT 10
          FOR SHARE SKIP LOCKED;

        ..DO WORK..

        ..UPDATE (IF NEEDED)..

        COMMIT;

• Batching and compression. The idea is to instead of picking the PENDING transactions one by one, pick a bunch of them (taking into account the size of the payload), and embed them together in a L1 transaction.

• The challenge with this feature is that it requires updating the method to extract the l2 transaction from the l1 data i.e. extract_data_p2shdd.

• This is almost compulsory for Ethereum, where the nonce causes lots of problems in the presence of concurrent transactions.